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ABSTRACT 

This study of 67 preservice and 67 inservice teachers’ performances on computer-based 
scenarios, prompting instructional decision-making and using embedded assessment, illustrates 
differences in how experienced and novice teachers make technology integration decisions. The 
major findings that emerged from the comparative analysis of novice and expert teachers’ 
exploration of the scenario and essay responses to its central question are that: (1) Inservice 
teachers addressed significantly more key elements of educational technology integration and 
implementation principles; (2) Inservice teachers’ essay responses were more focused on the use 
of technology as a learning tool and the importance of professional development while 
preservice teachers emphasized neutral topics such as the ubiquitous nature of computers and 
access to hardware and software; and (3) Both inservice and preservice teachers rarely 
mentioned assessment in their justifications. These results suggest that the teaching experiences 
of the inservice teachers influence their justifications for decisions. Experience counts. 

(Keywords: preservice education, computer-assisted instruction, computer-assisted assessment, 
technology integration, problem solving, decision making) 
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INTRODUCTION 

How can online, case-based simulations with embedded assessment be integrated into 
teacher education to develop, assess, and track preservice and inservice teachers’ technology- 
integration decision making? We will present data on how web-based simulations powered by 
the IMMEX™ problem-solving assessment software were utilized to capture how experienced 
and novice teachers differentially make decisions when infusing technology into instruction. 
Supported by a Department of Education Preparing Tomorrow’s Teachers to Use Technology 
(PT^) grant, two PT^ investigators have co-developed case-based, computer simulations of classic 
classroom technology implementation scenarios to enhance future teachers’ conceptual 
foundations and decision-making skills on integrating technology into the curriculum, called 
eTIPs problem sets. 

The complex nature of teaching challenges educators to make subtle judgments and 
agonizing decisions in daily classroom practice. Expanding instructional options with 
technology integration increases the complexity of the decisions teachers need to act upon. 
Skillful teachers anchor their practice in a combination of theory and praxis (Merseth, 1996). 
Case methods, when used appropriately, have the potential to bridge theory and practice 
(Shulman, 1992). Considering that preservice teachers lack the practice and are in the process of 
developing the theory, case methods could be used as a pedagogical tool to provide future 
teachers with experiences that mimic potential classroom dilemmas, including technology 
integration, that elicits active analysis, interpretation and the application of technology 
integration and implementation principles, early in their education, prior to actual classroom 
teaching. 
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Building on the IMMEX digital curriculum of case-based simulations for medical education and 
K-12 education, eTIPs problem sets for teacher education consist of three essential components: 
(1) a prolog — an opening scenario that presents a technology-integration challenge commonly 
encountered in “real” classrooms across the nation; (2) a problem space — menu items with data 
for research, analysis, and consideration; (3) an epilog — a summary of the potential logic a 
teacher could employ in solving the classroom dilemma. 

When teachers attempt to solve an eTIPs problem set, they logon with a customized ID 
and password (Figure I). The first screen encountered consists of the prolog in the main frame at 
the center and an accompanying problem space with primary menu options on the menu bar in 
the top frame (Figure 2). Based on how teachers conceptualize and frame the challenge and 
question posed in the prolog, teachers conduct a search in the problem space by selecting and 
analyzing individual menu items. Information on the simulated school for teachers to analyze 
and reach their decisions consist of teacher profile, student profile, school and classroom-based 
resources, with emphasis on technology, classroom configuration, classroom pedagogy, 
assessment, and scenario context, including information on student and teacher schedule, 
attitudes toward technology and professional development. As teachers execute their research in 
the defined problem space, the IMMEX problem-solving assessment system records each menu- 
item selection in chronological order. Additionally, the software generates a map that illustrates 
teachers’ step-by-step problem-solving approach, which can be used to guide assessment and 
engage learners in metacognition (Underdahl, Palacio-Cayetano, & Stevens, 2001). Upon 
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completion, teachers compose an online justification for their decision to integrate or not 
integrate technology (Figure 3). Teachers’ essays are scored based on the number of educational 
technology integration and implementation principles — eTIPs — addressed (Dexter, 2002). 



Figure 1— Step 1: Logging on to an eTIPs Problem Set with Individualized ID 
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Figure 2— Step 2: Interactive Problem Solving 
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Figure 3~Step 3: Composition of Essay 
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Figure 4— Step 4: Access to Search-Path Map and Problem Summary 
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PURPOSE AND OBJECTIVES 

Our goal was to conduct a comparative analysis of preservice and inservice teachers’ 
strategies and justifications on computer-based simulations to examine the impact of experience 
on technology integration and implementation decisions. Hence, the following two questions 
guided this research. 

□ What are the primary considerations both experienced and novice teachers make when 
deciding whether or not and to what extent to integrate technology into instruction? 

□ How do experienced and novice teachers differentially make technology integration 
decisions? 

THEORETICAL FRAMEWORK 

While rapid advances in technology and the proliferation of computers in schools have 
expanded pedagogical options for educators, teachers continue to struggle with how and when to 
integrate and implement technology effectively into instruction. According to Pierson (2001), 
schools are so eager to purchase and have teachers begin using technology, that they mistake 
simply having and turning on a computer as integration. However, merely knowing how to use a 
computer is not sufficient to ensure that teachers will effectively integrate technology into the 
learning curriculum. Instead, a teacher who effectively integrates technology is able to draw on 
extensive content knowledge and pedagogical knowledge, in combination with technological 
knowledge, when using technology in the classroom (Pierson, 2001). 

One challenge facing teacher education programs today is preparing teachers to use 
technology effectively in schools (Wedman & Diggs, 2001). Using online problem-solving 
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scenarios such as IMMEX can provide pre-service teachers with an appropriate context in which 
to construct knowledge about technology integration. These scenarios provide students multiple 
opportunities to learn to teach with, not just operate, educational technology around the 
following Educational Technology /ntegration and /mplementation Principles: (Dexter, 2002) 
eTIP 1: Learning outcomes drive the selection of technology. 
eTIP 2: Technology use provides added value to teaching and learning. 
eTIP 3: Technology assist in the assessment of the learning outcomes. 
eTIP 4: Ready access to supported hardware/software resources is provided. 
eTIP 5: Professional development is targeted at successful technology integration. 
eTIP 6: Teachers reflect on, discuss, and provide feedback about the role of and support of 
educational technology. 

IMPORTANCE OF THE STUDY 

Cases and case methods have been used as a pedagogical tool in preservice education for 
many years as far back as the mid-1980s (Merseth, 1996). Traditionally used in law, medicine, 
and in business, case methods are increasingly utilized in education to prepare teaching 
credential candidates to practice in a complex world where knowledge about teaching is not 
sufficient. Teachers and teacher candidates also need to know how to apply this knowledge in 
often complex and imperfect situations commonly encountered in the classroom. Case-based 
learning is an alternative method of instruction that enables teacher candidates to interpret 
complex situations that are in a constant state of flux and understand the theoretical issues 
involved. Analyzing case methods with frequently encountered classroom dilemmas in 

ERIC 
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technology integration engage credential candidates in complex cognitive processes that underlie 
successful performance in classroom settings and provide them with an opportunity to determine 
and apply suitable theory or principle to a given situation. Most importantly, preservice teachers 
can begin to acquire tools that will foster informed decision-making in situations where there are 
no easy, clear-cut answers. Since it is still uncertain how teachers make the decision to integrate 
technology or not and to what extent despite the significant investment in technology and 
professional development in recent years, case-based simulations with embedded assessment, 
like eTIPs problem sets powered by the IMMEX problem-solving assessment software, have the 
potential to capture and provide visual displays of key decisions teachers make in planning 
instruction with technology (Zhao & Cziko, 2001). 

Collaborating with both preservice and inservice teacher programs provided the UCLA 
IMMEX Project with a convenient sample and opportunity to contribute to research in teacher 
technology-integration decision-making, particularly addressing the following two questions. 
What are the primary considerations experienced and novice teachers use when deciding whether 
or not or to what extent to integrate technology into instruction? How do experienced and novice 
teachers differentially make technology integration decisions? Our observation that preservice 
teachers are inclined to focus on access to technology and marginally consider crucial 
dimensions of learning with technology such as technology implementation planning, 
prerequisite skills, and characteristics of learners, suggests curricular areas in need of increased 
emphases in educational technology curricula. Instructors can utilize data obtained from 
teachers’ performances on IMMEX eTIPs cases to design instructional interventions and refine 
curricular content for future educational technology courses. 
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DATA SOURCES 

The data for this study are drawn from 67 preservice teachers from a technology in 
education course at an urban university in California sponsored by the Department of Education 
PT^ implementation grant and 67 inservice teachers in a California Department of Education 
technology initiative administered by the California State University Chancellor’s Office. 
Preservice teachers solved the eTIPs cases as an assignment in their yearlong credential program. 
Similarly, inservice teachers performed the eTIPs cases as a requirement for a yearlong 
educational technology professional development program. Each teacher was supplied with a 
pre-assigned login ID and password and was given about 45 minutes to work independently in a 
computer lab setting. 

Unlike another research project that we are currently conducting in which the six eTIPs 
were introduced in the course of study prior to online problem solving, teachers in this study did 
not receive direct instmction on the six eTIPs or provided with literature on current research on 
educational technology integration and implementation principles. No specific instmctions were 
given to either group except to gather the data to make an informed decision and then discuss 
and justify their decisions in a written essay. Technology proficiency survey results from 
California Technology Assessment Project and IMMEX Project Evaluator’s survey confirmed 
that both preservice and inservice teachers had introductory or novice computer skills. 
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METHODS 

IMMEX automatically recorded the sequence of actions of each teacher while they 
performed a parallel case in eTIPs problem sets allowing quantitative and qualitative 
comparisons of the information acquired by the two groups. Additionally, each sentence in the 
teacher justifications about whether or not to integrate technology was tagged for key elements 
of the eTIPs addressed allowing quantitative comparisons of their decisions. The analysis of 
teachers’ essays enables instructors to learn the teacher’s decision and his/her rationale for it, as 
well as the depth of their reasoning, as judged by their coverage of key educational technology 
integration and implementation principles. 

Depending on the depth of a teacher’s written response, an essay can earn a maximum 12 
points, receiving up to two points for each of the six eTIPs. We used the following criteria to 
determine the essay’s score on each of the six eTIPs. 

• Zero— when the teacher makes no reference at all to the substance of the eTIP; 

• One— when the teacher makes a general reference to the substance of the eTIP, or implies 
consideration of the eTIP; and 

• Two— when the teacher discusses a specific eTIP comprehensively and provides supporting 
details from the information presented in the case. 

Sample Scoring of Teacher Essay 

“I intend to schedule a meeting with the other participating teachers so we can discuss possible 
objectives and to try to schedule our students for biweekly sessions in the computer room 
(Technology integration Planning/Scheduling: eTIP 1). I will contact my friend in another 
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district to see if we can begin having our students sharing information on a regular basis (Team 
Teaching: eTIP 6). I would like to have our students participate in a hunt for education 
informational websites. They would be competing with the fourth grade students in my friend's 
district. Initially we would just graph the number of available sites on a particular subject. We 
would then have a secondary contest to write reports about specific subjects and exchange them 
with the students in the other district. Some of the students would be able to send their 
information from home. Other students would have to use the computers at home. We would 
also begin the year with a general review of computer knowledge (Prerequisite Skills: eTIP 2) 
to be sure most students are aware of the basics and can also navigate the internet and can use the 
e-mail I would schedule local High school students to come and work both on reading skills and 
to also assist the students with their writing (Computer as Learning Tool: eTIP 2) and their 
sharing of information with the students in the other district. I would encourage the other 
teachers in my group to take laptops home and check on the progress of the students.” 

RESULTS 

The analysis of the search-path maps (showing teachers’ step-by-step search for 
information) and accompanying written essays revealed several insights into the users’ 
instructional decision making about technology integration. Both preservice and inservice 
educators examined the majority of the information in the simulation moving sequentially 
through the menu items (Figure 1). While there were few item-selection differences between the 
groups, an independent samples T-test showed that inservice teachers addressed significantly 
more eTIPs in their essays than preservice teachers (p < .01). The analysis of essays for score on 
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each eTIP also showed a differential mention of technology integration topics between the 
groups. Inservice teachers focused significantly more on eTIP 2, the use of technology as a 
learning tool (p < .05), and eTIP 5, the importance of professional development (p < .01) while 
preservice teachers were more likely to mention more neutral topics such as the ubiquitous 
nature of computers, impact of technology on teaching, etc. Both groups equally mentioned 
hardware, software and standards. Interestingly, the mention of eTIP 3, assessment, was rare for 
either group. A listing of the most frequently mentioned issues is shown in Table 1 . These data 
indicate that although the two groups accessed the same data from the simulations, the 
experiences of the inservice teachers influenced their perception of the simulations and their 
written decisions. 



Figure 5 — Search-Path Maps for Inservice and Preservice Teachers 
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Table 1 — Most Frequently Mentioned Issues 





Preservice 

(Frequency) 


Inservice 

(Frequency) 


Technology Implementation Planning/Curriculum 


11 


35 


Computer as Learning Tool 


14 


28 


Prerequisite Skills 


14 


20 


Cooperative Learning 


10 


20 


Professional Development 


4 


12 


Technical Support 


8 


15 


Linking Curriculum to Technology 


0 


6 



CONCLUSION 

While unraveling how teachers make decisions to teach with or without technology is 
only a single component of developing teachers’ technological competencies and instructional 
platming for technology integration, the impact can be far reaching. The ability to use eTIP 
IMMEX-powered simulations to capture variations in the quality of both preservice and 
inservice teachers’ considerations and decisions for technology integration in a range of 
classroom environments by electronically reconstructing teachers’ decision-making processes 
and coding teachers’ written justifications for alignment with research-based educational 
technology integration and implementation principles provide support for eTIPs simulations as a 
viable instrument for developing and assessing teachers’ conceptual knowledge of technology 
integration. The striking difference between the quality of the preservice and inservice teachers 
justifications for their decisions despite the similarity in their technology skills based on self 
reported surveys of technology skills suggests that teaching experience is one of the most 
significant influences on the quality technology integration. The higher incidence of key 
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elements of educational technology integration and implementation principles in inservice 
teachers’ essays was not accidental occurrence, but an experiential advantage. Experience 
counts! The infrequent mention of the assessment potential (eTIP 3) of technology by both 
preservice and inservice teachers most likely is a reflection of the limited use and access to 
technology for assessment in current classroom practice. 

These findings can help to shape the quality of interventions in professional development 
and teacher education courses so as to maximize the level of technology integration and 
implementation capabilities in K-16 educators. Instructors can utilize data obtained from 
teachers’ performances on the IMMEX-powered cases to design instructional interventions and 
refine curricular content for future educational technology courses. A deeper understanding of 
how teachers prioritize issues and make decisions about their instructional programs not only has 
immediate importance for current training programs, but also has implications for future 
directions of teacher education programs, creation of technology initiatives to enhance preservice 
and inservice teachers’ technological competences, and school-site professional development in 
technology. Case-based simulations coupled with embedded assessment, like eTIP IMMEX- 
powered simulations, that can identify, develop and assess how teachers make decisions about 
designing instructional programs may hold promise for accelerating this understanding how 
teachers make the decision to integrate technology or not into instruction. 
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